NOfIGE:  When  governinent  or  other  drawings^  speoi- 
fiGatiofls  or  other  data  are  used  for  any  pui^ose 
other  them  in  eonneGtion  \^th  a  definitely  related 
government  procurement  operation,  the  U.  S. 
Government  thereby  incurs  no  responsibility,  nor  any 
obligation  x^atsoever;  and  the  fact  that  the  Govern^ 
ment  may  have  foimilated,  furnished,  or  in  any  way 
supplied  the  said  drawings,  specifications,  or  other 
data  is  not  to  be  regarded  by  implication  or  other'^ 
wise  as  in  any  manner  licensing  the  holder  or  any 
Other  person  or  coiroration,  or  conveyL^  any  ri^ts 
or  peiSDission  to  manufacture>  use  or  sen  any 
patented  invention  that  may  in  any  way  be  related 
thereto* 
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in  I^Ib  sys^slum  ve  would  like  to  discuss  soine  of  the  experiences  ve  have  had 
in  Installing  and  fflaintalnlng  the  System  Training  i^dgram  for  the  Aircraft  Con* 
trol  and  Warning  Radar  Net  of  the  Air  Defense  Conanand.  We  believe  that  the  hu* 
man  factors  problems  we  have  encountered  in  the  five  years  since  the  training 
program  was  first  instedled  are  representative  of  those  to  be  found  in  all  Sim* 
iiar  systems. 

Although  all  of  the  participantB  in  this  synposium  have  been  involved  primarily 
with  the  research  effosrt  associated  with  the  System  Training  Program>  all  of 
us  have  had  considerable  experience  in  the  field;  "on  the  firing  line"  as  it 
were;  applying  the  conceptual  training  model;  confeirlng  with  military  person* 
nel;  and  adapting  training  aids  and  procedures  to  operations  problems  as  they 
arise.  Each  Of  the  speakers  will  discuss  a  different  aspect  of  the  problem  of 
system  training.  I  will  describe  the  Air  Control  and  Warning  net;  the  AC&W 
System  of  the  Air  Defense  Caanmtuid;  and  discuss  the  training  retirements  of 
such  a  system.  Dr.  Rogers  will  discuss  simulation  technology  and  training  in* 
puts.  Dr.  Jordan  will  talk  about  knowledge  of  results  and  how  it  is  utilized. 
Dr.  Jensen  will  discuss  the  military  cultural  environment  and  its  liig>lications 
for  training.  Dr.  Ford  will  indicate  how  the  principles  and  techniques  we  have 
learned  may  be  applied  to  training  man*machlne  systems  of  the  future. 

Tbs  AC&W  system  is  a  complex  man*machlne  Infoznatipn  processing  system.  Its 
ultimate  mission  is  to  defend  the  United  States  against  air  attack.  The  goal 
of  the  training  program  is  to  increase  the  proficiency  of  the  human  components 
of  the  system  in  aceoapllahli^g  this  mission. 

For  those  of  you  who  may  not  be  familiar  with  the  system;  let  me  tell  you 
briefly  how  it  works  by  describing  a  typical  element;  the  Direction  Center. 

The  Direction  Center  receives  information  in  many  forms.  Although  its  primary 
input  is  radar  data;  it  also  receives  flight  plan  information;  weather  infor* 
matlon;  and  intelli^nce  infozmation.  It  uses  this  information  to  identify 
all  aircraft  which  appear  in  the  air  space  for  which  it  is  responsible.  Iden* 
tification  is  usually  accomplished  by  eeiipirlmg  the  position;  speed;  direc* 
tion;  and  altitude  of  an  aircraft;  obtained  troa  radar  data,  with  similar  in* 
formation  obtained  from  flight  plan  data.  If  it  is  not  possible  to  correlate 
flight  plan  and  radar  information;  an  interceptor  aircraft  is  scrmi^led  and 
directed  to  the  location  of  the  unknown  aircraft  so  that  positive  visual  iden¬ 
tification  may  be  made.  In  order  to  accomplif^  the  air  defense  mission  it  is 
necessary  for  a  Direction  Center  to  cooperate  with  adjacent  stations  by  pass¬ 
ing  relsvant  information  back  and  forth  and  to  keep  tbs  coDmsnd  personnel  in¬ 
formed  of  its  actions. 

The  early  version  of  the  AC&W  net  was  called  the  Hanual  Air  Defense  ^tem  be¬ 
cause  all  phases  of  its  operations  were  carried  on  primarily  by  hunans.  Be¬ 
cause  of  the  ever  increasing  density  of  air  traffic  over  tte  contj^ntal  Uhlted 
States  and  because  of  the  di^lo^pt  of  new  vespons  systems  which  reduce  the 
amount  of  available  varaing  tte  for  defense;  the  AC&W  net  has  teen  autcoated. 
Ths  presant  air  defense  system;  SACS;  has  at  its  core  a  high  spate  digital 
computer.  tte  present  system  the  computer  handles  most  of  the  routine  data 
processing  finietions.  IteMna  aoxUtor  and  assiat  the  cmoputer  and  make  the  more 
ij^rtent  decisions  regarddng  threat  evaluation  and  tactical  action. 
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systan  which  I  have  been  deacribing  to  you  Is  a  laxge  scale;  complex;  ln« 
foxnatlon  processing;  nan-machine  system.  It  is  large  scale  and  complex  in 
the  sense  that  it  has  a  task  which  has  naxvy  dimensions.  The  task  cannot  be 
pexToziaed  by  a  single  individual  or  by  a  single  machine;  but  requires  the  co¬ 
ordinated  actions  of  many  men  and  machines.  Tb»  field  within  which  the  people 
in  the  system  operate  is  so  large  that  no  one  of  them  Or  no  group  can  have 
access  to  or  assimilate  all  the  infoznation  which  is  available.  In  most  cases 
no  individual  can  appreciate  all  the  raskiflcations  of  any  action  he  nay  take^ 
Consequently;  there  oust  be  fractionation  of  the  decision-naklag  proeesses  and 
some  over-all  supervision  and  coordination  of  the  decisions  made. 

The  Gcomodity  with  which  the  system  deals  is  information.  The  system  operates 
by  coding;  storing;  decoding;  and  reorganizing  infoimatlon  for  many  different 
purposes.  The  essential  pre-reqalelte  for  adequate  iystem  functioning  is  the 
rapid;  correct  manipulation  and  transmiaaion  of  infoziaatiOn  aaoag  the  varlOua 
ay  item  cooponente. 

What  are  the  training  needa  for  a  complex;  man-machine;  information  procaaaing 
system  such  as  this?  The  first  retirement  is  to  develop  eempensnt  skills  of 
individual  operators.  The  Air  Training  ComaBad  maintains  schools  in  which 
operators  learn  about  air  defense  operating  procedures;  leara  bov  the  ecmqiuter 
wozkS;  and  how  to  operate  the  varioua  consoles  and  telephone  networks  by  which 
they  cenmunleate  with  the  computer  end  with  each  othar. 

A  aecond  requirement  is  a  period  of  on-the-JOb  training  at  a  Direction 
Center  in  which  the  trainee  begins  to  aj^ly  what  he  has  learned  in  school  and 
begina  to  operate  as  an  Integral  part  of  his  crew.  A  third  requirements  Is 
system  training  and  it  la  this  latter  requirement  with  which  we  are  primarily 
concerned  here. 

System  training  seeks  to  dtvelop  the  skills  by  which  individual  crew  BMonbers 
work  together  smoothly  as  an  efficient  team.  It  augments  Individual  component 
skills  by  devtloping  such  interactive  skills  as  pacing;  load  balancing;  ezror 
checking;  and  information  filtering.  It  provides  the  o]^rtunlty  for  system 
personnel  to  practice  together  in  dealing  with  realistic  operational  threat 
■Ituatloni.  . 

The  coBqputcriBfd  syatem  environment  introduces  unique  training  prohlmma  to 
those  of  us  who  have  been  concerned  with  the  human  factors  reqairements  of  sys* 
terns.  Lst  me  state  briefly  what  some  of  these  problem  areii  are.  The  speakers 
follow  will  discuf  i  thsm  ^  more  detail. 

Btia^us  innuts.  A  system  traix4z^  exercise  must  accomplish  two  purposes: 
First;  it  muii  provide  the  crew  with  the  p^rtwiity  to  experience  practice 
defending  against  various  kinds  of  potential  air  threat  situations;  second;  it 
must  laiPM  load  on  the  Infozwtion  procassing  cod  deciiion  aakizm  operations 
of  the  (joritma.  The  slmula^d  air  picture  is  one  of  the  priairy  stli^us  sgea# 
cias  in  a  system  training  exercise  for  aopoB^iehing  these  purpoiee. 

Motiyigbion  end  Morale.  One  of  the  main  charaoteriitics  of  the  SAI^  systra  if 
the  plyd^ldglcal  isdlation  of  the  eperatpra.  Almoet  aU  interpersonal  rela» 
tioos  auuit  be  conducted  by  a  telephcne  or  via  tbe  computer.  Ad^  to  this  is 
the  fact  that  in  order  to  perform  adequately  at  the  speeds  reqaired;  it  is 


xiecessary  to  divide  the  various  tasks  late  sniall  units,  l^se  two  factors  com¬ 
bine  to  ^Veat  the  ladlvldual  operator  from  gaining  an  over-all  picture  of  hov 
the  system  operates  aad>  more  impcrtaatiy,  how  the  fun#tloas  he  perfoiiss  con¬ 
tribute  to  the  system  mission.  Coasequeatly>  It  Is  dlffieult  for  crew  personnel 
to  maintain  at  a  high  level  the  motivation  re(|ulred  to  perform  the  complex  oper¬ 
ations  Involved. 

Another  problam  in  the  area  of  motivation  and  morale  Involves  the  usuxpiatlon  by 
the  computer  of  many  of  the  functions  which  have  previously  been  considered  the 
particular  province  of  the  human,  ^re  Is  a  general  feeling  among  ell  system 
personnel  that  they  are  not  in  to  vch  with  the  real  environment  and  can  only  get 
information  concerning  this  environment  by  means  of  a  complex  gadget  which  they 
do  not  completely  understand  and  which,  to  some  extent,  they  regard  with  some 
degree  Of  fear.  In  addition  to  this,  there  is  the  feeling  of  loss  of  control 
over  the  environment.  Moat  of  the  .decisions  they  make  must  be  implemented 
through  the  computer  which  is  set  up  to  amplify,  and  In  some  cases  modify,  the 
results  of  these* decisions. 

Iteovrledae  &£  Itosults.  System  training,  like  all  other  forms  of  training,  re- 
q;ulreer  tiie  provision  of  adequate  knowledge  of  results'^  The  requirements  for 
fed  had:  Information  differ  for  various  people  In  the  system  and  for  the  teem 
as  a  Whole.  Since  these  requirements  changs  as  a  function  of  the  threat  situ¬ 
ation  end  the  developing  proficiency  of  the  crew,  the  problem  of  providing  ade¬ 
quate  knowledge  of  results  resolves  Itself  Into  a  problem  of  selecting  approp¬ 
riate  Information  at  the  right  time.  Too  much  Ixxfozmatlon  may  be  as  bad  as 
none  at  all.  Id  providing  knovled^  of  resulti,  the  trainer  must  determine  an¬ 
swers  to  qiMstlona  such  as  the  following:  Sow  much  information  should  be  pro¬ 
vided?  vian  should  it  bs  provided?  T3m  should  It  be  collected  and  dlsplsyed? 

Bov  should  It  relate  to  the  needs  of  the  trainees  as  these  change  over  tine? 

utilization  o£  Khowle^  o£  Results.  In  System  Training  Progrcn  exercises, 
trainees  are  giv^  infotmatlon  about  the  simulated  air  threat  with  which  the 
system  was  faoed  and  a  description  of  how  the  system  performed  in  meeting  the 
threat.  This  information  is  provided  to  the  crew  for  debriefing,  la  the  de¬ 
briefing  session  the  craw  attempts  to  Identify  sreas  of  Inaffielent  operation 
and  to  produce  solutions  to  the  problems  thus  Identified.  These  solutions, 
which  usually  consist  of  modified  aerating  prooadurea,  are  then  tried  out  in 
a  later  ixerolse  and  may  be  adopted  as  coanend-widt  ■taadard  optratlag  preoadure. 

gersonael  flelertiw  and  Turnover.  The  System  Training  Frogren  is  not  ImiMdi- 

probllmr^  personnel  selection.  These  arc  hsndltd  in 
acoordonee  with  Air  Force  policy.  The  system  must  operate  with  the  persOimel 
with  v^Ch  it  is  i^ovidsd,  and  the  training  prognss  amst  do  the  seme .  How¬ 
ever,  of  vital  concem  are  ^  training  prpblsBui  Introduced  by  perspnoel  turn¬ 
over.  One  of  the  goals  of  training  Is  to  produce  a  veH-lntigratad,  smoothly 
funetioning  tenn  in  wbieh  the  individual  teon  amiilbers  work  well  together. 

Ccnataat  turnover  of  personnel  nitigatei  sgainit  this  and  reduces  ths  sff Icisncy 
of  ths  systsBn  as  a  whols.  Gns  of  ths  reguirsmsnts  of  a  systam  tralniag  progrsm 
is  to  ds^l^  tsehni^s  which  viH  rsduos  ths  dsletsrious  effects  of  parsohnel 
turnover. 

These,  briefly,  sre  some  of  the  syetsm  training  problaa  areas  we  have  identi¬ 
fied.  Bow  let  me  describe  bow  a  systms  training  exercise  is  conducted  so  that 
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you  nleht  have  a  greater  appreciation  of  how  the  training  techniques  which 
the  following  speakers  i:ili  dlseues  fit  in.  All  system  training  exercises 
are  conducted  at  field  operational  sites.  The  exercises  usually  last  ahout 
two  hours  exclusive  of  the  debrief lag.  Team  personnel  are  exercised  in  their 
operating  positions  using  all  the  equiianent  whi(±  they  would  normally  use  in 
real  operations.  The  threat  situation  is  developed  la  consi^tatlon  with  Air 
Force  operations  and  training  officers  end  is  presented  to  the  crew  by  means 
of  nonaal  operational  input  equiinent.  Although  the  ii^ts  to  the  system  sim¬ 
ulate  a  real  air  situation,  it  is  difficult  for  system  personnel  to  distin¬ 
guish  between  the  sianileted  and  real  inputs.  Except  for  the  fact  that  system 
personnel  know  that  the  air  situation  is  part  of  a  training  exercise,  they 
^  required  to  perform  exactly  as  they  would  in  the  ease  of  a  real  air  threat. 
System  training  exerciees  have  been  designed  for  funetlonal  units  of  the  SAGE 
system  of  various  sizes  from  the  single  Direction  Center  to  the  entire  nation- 
vide  network  operating  slmaitaneously. 

Durlag  the  exereise  perfoimanee  data  are  collected  for  the  provision  of  loiow- 
ledge  of  results*  Since  all  of  the  inputs  are  designed  beforehsad,  the  trainer 
bae  casvlete  knowledge  and  control  of  the  stlmulue.  After  the  exercise,  crew 
pereoanel  meet  together  for  a  debrloflng  seesien  in  which  the  exercise  is  dls- 
cueeed. 
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^  tbe  (mtildtF,  oaa  of  the  aoit  lJi(pr«6«ive  aij^eti  of  i^tw  training  li  tha 
axtant  to  vMeb  tb*  enrLKOBent  of  tbe  iyitiH  baa  boon  aimlatodi  Vo  not  only 
iliulato  a  coox^natod  dynaale  air  pieturo>  vo  alao  tljauiato  infozaation  eon* 
eoniing  this  air  pletox*  ttcm  tbo  Fidoral  Aviation  Aioney>  adjaeont  Mlltary 
ualti,  highor  boadquartora,  and  intalliganea  aoureaa.  Portbaxapra,  va  aimlata 
air  baaaa>  figbtara  in  idia  air,  and  ovon  vaatbar,  at  tlJMa.  All  in  aU,  va 
try  to  aJjiulata  all  aouroaB  of  inforaatlon  lAdeb  aight  raaaoaably  affaet  tba 
parfoniaaea  of  tba  vudt  baing  trainad.  dftan  oaa  of  tba  firat  foaitlona  aakad 
ia:  *'VlQr  do  yon  alaidatot  Vby  not  naa  'liva*  iBgata  and  'liva*  airoraftT"  onr 
agparlanea  baa  ylaldad  tba  foiloving  anavara: 

!•  Tba  firat  raaaon  for  alinilatiOB  la  in  ordar  to  praetiea  rara  avanta.  Tba 
Air  bafanaa  syataa  Training  Prograa  la  intandad  to  balp  tba  Air  bafanaa 
Byataai  prwptn  to  dafaat  tba  airboma  aapaota  of  an  attaeb  on  tbia  eouatry. 

Va  bopa  tbia  avant  vill  navar  occur.  If  it  doaa  occur,  tba  ayataa'a  par« 
fOnmnea  tdll  bava  to  ba  parfaet  tba  firat  ttna  or  it  will  ba  too  lata  to 
uaa  tba  raal  avant  aa  a  baaia  for  trainlngi  Tbarafora,  tba  only  vay  tha 
ayataai  can  practioa  in  viidi  a  alwilatad  air  attack.  Altbouidt  tbia  raaaon 
■ay  aaaa  uniqna  to  tba  Dafanaa  syataa,  it  la  fair  to  acy  that  any  ayatan 
■ay  ba  raqairad  to  bandla  aituatlona  vbicb  occur  infravcantly  la  raal 
aapanaaca^  Sy  aiiBilatlon,  tba  ayitan  can  praetiea  daa^ing  vitb  tba  rara 
avanta  and  tbaraby  laam  to  bandla  tbaa. 

Si  Anetbar  raaaon  for  alia^tioa  la  ita  raiatlvaly  lov  coat.  In  ordar  to 
train  a  ayat«a  it  la  nacaaaary  to  rapaat  tba  aituatlon  to  ba  laarnad  aany 
tlMa.  If  tba  taak  to  ba  laarnad  la  launcblag  a  aiaalla,  aacb  rail  launch¬ 
ing  oBuaily  coita  oaa  aiaaila  and  rapatitlon  for  training  oaa  baooM  arpan- 
alva.  On  tha  other  bandi  al^ugb  tba  coat  of  a  aiaalla  alanlator  alght 
ba  aavaral  tlaia  aa  aueb  aa  tba  aiaalla,  tba  alanlator 'a  coat  can  ba 
proipratad  ovar  a  largo  auabwr  of  aiaulatad  lanacblnga.  Tbua,  tba  ovar- 
aii  coat  of  training  by  alaulatlon  can  ba  oonaldarably  laaa  than  training 
vitb  tba  "raal  thing."  In  8TP  va  uaa  fllaa,  aagnatlo  tapaa  uA  aorlpta 
In  plaea  of  liva  aircraft.  Wo  ara,  tbarafora,  abla  to  rapaat  tba  aaaa 
atiilatad  attack  aa  oftan  aa  daairad  for  tndnlng  puzpoaaa  at  far  laaa 
than  ^  coat  of  duplicating  tha  training  by  raiQ  aircraft. 

3.  A  third  raaaon  for  alaulatlon  la  tba  fact  that  tba  Input  to  tba  ayataa 
baing  trainad  can  ba  oontrollad  by  tba  tralaing  apaolallat.  Ba  can  vary 
a  ayMlatad  input  frqa  aaarclaa  to  aaarclaa  according  to  bla  tealMag 
atratasy.  In  order  for  laarniag  to  occur,  a  problaa  aituatlon  auat  ba 
craatad.  iba  crav  baing  tmlnad  auat  find  itaalf  unnbia  to  pwnfbni  na 
vail  aa  it  daalrta  in  worn  altwtion.  For  anaivla,  tba  air  aarvaillanoa 
aaotion  of  an  air  dafania  unit  cwi  ^  feroad  to  pat^xm  in  auob  n  altuntlon  ^ 

by  aixvly  lanrtaalng  tba  aaount  of  panatraitlai  idr  trafflo.  It  la  ra« 
lattvi^  anay  to  produoa  wak  hoary  traffle  ieaia  in  STP  taareiaan  vbarann 
varlnbla  eat  hardly  M  eontrollad  at  ali  la  rani  llfa. 

Tbia  prlaelplf  of  ooatrplllag  tbo  Input  to  produca  a  laarniag  aituatlon 
ean  ba  tataadad  vitb  alaulatlon  tac^quoa  to  parta  of  tba  ayataii  not  In 
oontact  vitb  tha  outalda  vorld.  Tbua  aalactlva  tnl^ng  can  ba  pr^dad 
for  aubayBtaaa  vblla  tboy  art  oparatlng  In  ooataxt  of  tba  total  ayataa. 

Oaa  aaii^  of  tbia  in  ^  Air  jpafUnaa  fbrataa  vould  ba  training  aatrolaaa 
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dealgaed  to  ei^hasiza  tralalog  of  tbs  Identification  Section.  This  sec- 
tion  baa  the  task  of  con^ctiy  aaeoelatlng  tracks  representing  aircraft 
in  the  air  and  air  movements  data  vhleb  aratbe  intended  flight  paths  pro¬ 
vided  by  the  aircraft  operators.  To  piece  the  Mentlflcation  Section  in 
a  problem  situation,  the  training  specialist  must  increase  the  number  of 
tracks  to  be  correlated  end  Increase  the  number  of  items  of  air  movaDents 
data.  This  can  be  aceomplisbed  m  STP  by  simply  prOrlding  the  Air  Sur« 
veillanee  Section  with  scripts  so  that  it  can  process  perfectly  an  STP 
exercise  with  a  lot  of  penetrating  traffic.  Since  penetrating  traffic 
must  be  correlated  vlth  air  movements  data,  the  Identification  Section 
Is  now  faced  with  a  task  as  hard  as  it  can  ever  be  expected  to  perform. 

If  the  section  cannot  iamediately  adjust  to  the  load  it  now  has  reason 
for  learMng  and  an  opporttmlty  to  observe  its  progress  independently  from 
that  of  the  Air  Survelllenee  Section.  As  you  can  teH  from  the  smount  of 
time  I  have  spent  on  this  point,  control  of  the  input  is  perhaps  the  most 
Important  reason  for  simulation  in  a  training  program. 

4.  Another  reason  for  simulation  is  that  when  simulation  techniques  are  used, 
a  very  precise  knowledge  of  the  input  to  the  system  is  available  to  the 
training  specialist.  This  precise  knowledge  of  the  input  is  fundamental 
to  his  ability  to  provide  accurate  knowledge  of  results.  The  Importance 
of  accurate  knowledge  of  results  will  be  discussed  by  another  speaker. 

Given  these  purposes  for  simulating,  the  next  question  is:  "Hov  realistic 
must  this  Slnalatlon  be?"  In  our  experience  we  have, provided  Simuiated  inputs 
which  have  ranged  frc»a  extremely  crude  scripts  to  a  ^ry  precise  duplication 
of  the  real  thing,  ^ior  to  over  experience  with  the  Air  Defense  System,  the 
answer  was:  "Simulation  should  be  adequate  for  training."  This  answer  gener¬ 
ated  the  following  corollary:  "The  more  realistic,  the  more  adequate."  This 
particiilsr  point  of  view  seems  to  be  characteristic  of  the  trainees  and  their 
supervisors  during  the  early  stages  of  crew  develojnent  where  any  slight  dev¬ 
iation  from  realism  will  be  used  as  an  excuse  for  avoidixig  real  problems  and 
real  Issues  which  the  crew  onist  solve  before  it  can  develop.  On  the  other 
hand,  our  training  specialists  in  the  field  have  come  to  the  positipn  that 
each' east  of  simulation  must  be  considered  on  its  own  merits  sad  la  relation  to 
what  Its  training  objaetives  are.  Haereforsf  the  question  must  be  restated  as: 
"How  is  this  proFOsed  msthpd  of  simulation  going  to  help  train  whom  to  do  what? 
It  has  been  our  experience  in  the  laboratory  tlmit  this  latter  approach  is  cor¬ 
rect  and  the  corollary  mentioned  above  it  not  nscsssarlly  so.  A  general  rule 
of  thumb  vs  have  u#ed  is,  simalation  should  be  rsallstio  enou|d)  so  that 
ths  peopls  involved  ^  the  system  being  trained  arc  required  to  perfom  similar 
futtctlons,  slmilnr  responses,  sad  undergo  aMtusl  Interactions  similaur  to  those 
required  la  real  operations."  LUm  all  rules,  this  oas  has  its  exceptions, 
purtieulKrly  where  more  realism  is  required  for  aotivatlpa  or  evaluation.  The 
exact  da0ee  of  realiim  still  remains  a  prpfsssioiuQ  Judgnent.  In  ether  words, 
"rtalimii"  la  not  the  sasyer,  or  really  even  the  question.  ^  prohlsm  is, 

"Wl^t  if  hfit  to  aceomplish  the  needad  training?"  This  problsB  can  best  be 
solved  by  the  wise  judgmsat  of  an  e^q^rlenced  professional  tralnl^  specialist. 

Several  gaaeral  prineiplet  regarding  simulation  have  arisen  from  our  experience 
Among  these  are; 

1.  Before  undertidplng  to  provide  elmalatipn  adoquate  for  training,  it  is  nec- 
iiiary  to  acquire  a  thorough  knowledge  of  the  system  to  be  trained.  We 


iBufli  know  who  ddes  vh&t  before  we  can  decide  which  slntalatlon  ntethod  Is 
best,  f^ferebly,  the  system  should  be  studied  in  o^ratlon.  If  this  Is 
not  j^sslble^  Its  plans  must  be  intensively  analyzed^  their  consequences 
predicted  as  far  as  possible,  and,  if  time  is  available,  a  "breadboard" 
mo^rup  of  the  system  should  be  built  and  placed  in  operation  in  a  labors 
atbry  before  the  final  simulation  method,  materials,  etc.,  are  produced. 

2.  Aa,  accurate  knowledge  of  the  events  being  sionilated.  is  fundamental  to  any 
^O'ltten  or  pictorial  nateriaLs  describing  these  events.  system  train¬ 
ing,  we  have  found  it  desirabie  to  provide  the  personnel  responsible  for 
collecting  data  for  knowledge  of  results  with  charts  showing  the  "true" 
air  picture  every  five  minu^S.*  IMS  precise  knowledge  of  the  Input  Is 
designe'd  to  help  these  people  detezmlne  what,  If  anything,  la  going  t/rong 
at  any  given  time.  Inaccuracies,  ia  such  printed  descriptions  create  er¬ 
roneous  Ifflowledge  of  results.  These  errors  are  used  by  the  trainees  to 
avpld  being  trained  ,,just  aa  lack  of  reallsii  is.used  In  early  stages  of 

.  .  crew  devalopment. 

i..  VThen  agencies  interacting  with  the  aystem  by  means  of  people  being  s^ 
ulated,  tM  . people  must  be  simulated  accurately*  .In  dp-  case,  interceptor 
pilots,  Air  Route  Traffic  Control  Center  personnel,  a£r  base  personnel, 
ate.,,  must, be  accurately  . simulated  because  .person-to-person  interaction, 
eapeeji,ally  tbs  kind  of  task  oriented  pacing  that  takes  place  over  telephone 
ax^or  radio  coomiunlcations.  Is  always  critical  to  the  performance  of  the 
ma^r  of  ^  system  being  trained. .  Mpx^pbr  simulation  of  these  kinds  of 
ih^ractloaa  can  ;lead  ;to  aybidance  of  training  .but  more  importantly,  it 
can  lead  to  haimful  .effects  such  as  the  use  of  Improper  radio-telephone 
procedure,  habitual  failure  to  request  important  Infoxuatlon  because  the 
ainulcitor  volunteers  It,  and  failure  to  check  infPzttatlOn  because  the  slm- 
xilator  Is  always  correct.  Finally,  Improper  simulation  of  personal  inter¬ 
actions,  can  most  readily  lead  to  a  poor,  attitude. tpwsird  training  such  as, 
"So  what,  it's  only  a  gamer' 

k.  On  other  hand,  it  is  possible  to  carry  accurate  simulatipn  too  far. 

In'  our  case,  it  was  found  rather  easy  to  develop  an  elaborate  table  of  kill 
p^babilities.  These  probabilities  could  be  made  to  take  account  of  all 
sorts  of  possible  events  in  the  conduct  of  an  intercept  after  the  pilot 
had  instructed  to  splai^  the  tost  lie  target.  The  resulting  slmula- 
tito  tables  could  be  calculated  to  three  decimal  points*  It  is  Obvious 
ttot  this,  degree  of  accuracy,  is  quite  irrelevniit  if,  in  the  course  of  a 
training  ndsalon,  any  one  intercept  director  has  less  tiian  tan  Intarcapta 
tp  be  evaluated.  In  other  words,  by  vlrtua  of  other  oapeota  of  the  train¬ 
ing,  program,  the  three  dsctmal  plmcas  and  a  good  deal  of  the  rest  of  the 
soptostleatipn  of  the  kill  probability  tables  was  wasted  since  it  did  not 

.  ,  swim  any  dlfferenca  to  anyone  being  trained.  :  be  fact,  since  it  so  often 
resulted  in  an  iatareept  director  being  told  that  an  othervise  parfact  in- 
tarcept  failed  due  to  the  kill  probability,  the  resMting  daclihe  in  mo¬ 
tivation  has  challengad  the  whole  notion  of  a  kill  probability  table. 
Ptofessimwl  Judgment  has  yielded  a  siapllfled  simulation  where  the  con- 
duet  of  the  Intercept  is  svaluated  up  to  the  firing  pass  and  e  few  numbers 
written  on  enottor  aid  are  used  as  a  crude  kill  probability  table  tp  ac¬ 
count  for  tto  final  outecoe.  Thus,  the  rest  of  the  system  has  a  realistic 
outegme  fed  back  into  it  and  the  intercept  directors  ere  net  penalized  for 
events  beyond  tteir  control. 


Fl^blllty  l8  another  key  principle  of  simulation,  If  simulation  Is  to  con¬ 
tribute  appropriately  to  a  training  program.  Just  as  training  needs  will 
vary  from  day  to  day,  crew  to  crew,  subsystem  to  subsystem,  the  sinn^ation 
teehniques  must  be  permitted  to  co-vary  or  they  will  fall  to  achieve  their 
goal  of  tralxd.ng.  this  approach  leads  to  a  proposition  t^leh  is  true  in 
general,  but,  of  course,  has  some  exceptions,  to  wit:  "If  the  simulation 
task  is  repetitious  and  requires  precision,  try  to  get  a  machine  to  do  it; 

In  ell  other  cases  a  person  is  preferable  because  his  procedvores  can  be 
changed  overnight. "  A  strict  application  of  this  i^inolple  will  often  lead 
to  the  apparently  absurd  situation  where  there  are  more  people  on  the  simu¬ 
lation  team  than  on  the  operatlona  unit  being  trained.  This  anODSly,  however 
appears  to  be  one  of  those  "feots  of  life"  which  might  as  well  be  accepted. 

We  have  found  that  premature  autGnatlon  of  simulatlra  in  an  attempt  to  reduce 
the  number  of  simulation  personnel  has  been  one  of  the  prime  sources  of 
difficulty  In  our  training  program. 

In  aumam^,  let  me  atate  the  mrinoiple  facts  we  have  learned  concerning  the  eontrl 

hutlon  of  simulation  techniques  to  a  training  pro^iram: 

1.  Simulation  is  needed  in  order  to  provide  a  syatem  with  practice  in  handling 
events  which  occur  infrequmstiy  during  operations. 

2.  It  Is  relatively  inexpensive  because  its  initial  coat  can  be  pro-rated  ever 
a  number  of  repetitions. 

3.  Most  Importsnt  of  all,  simulation  provldis  tbs  training  specialist  \dth 
control  of  the  Input  to  the  systsm  so  that  hs  can  easily  laplemant  a  training 
strategy  of  hie  chooelng. 

b.  The  question  of  realism  is  really  Irrelevant.  The  proper  criterion  of 
aiaulation  IS  a  professional  Judipasat  of  how  effsotlvsly  It  ehgbles  a 
training  goal  to  be  achieved  by  a  specif le  system,  subsyitam,  or  eonponent. 

$.  Before  any  simulation  is  atteaptsd  it  is  neoessexy  to  know  thoroughly  the 
system  to  he  trained. 

6.  Znaoourate  slaua.ation  aids,  materials,  and  taohnlquss  usuaaiy  dafeat  the 
pwrposa  of  training  by  providing  the  orew  with  an  iKouaa  for  avol^ng  ita 
own  problama. 

7.  Too  aophlatloatad  aiaulatiM  can  diatraot  a  orev  from  facing  raid,  laauas. 
Inataad  they  may  apand  thair  tima  quastionlng  the  facta  undarlylng  the 
aiaulation.  Profateional  dudgaant  la  raquirad  to  aat  the  propar  laval  of 
sophistication  for  training  purposss. 

8.  Finally,  siBadatlon  must  ba  flaxibla  if  it  if  to  ppntrlbuta  to  a  training 
program. 


'MWoMACHIiaS  IRAINIII}  TBC^QUES  ^  TORPS,  tmmCKj  ASD  DEBRIlFPS" 

H^emlah  JorAan 


*  9  ‘ 


As  you  oust  have  noticed  >  we  have  developed  a  private  language  for  talking  about 
this  program.  I^ee  words  that  we  use  quite  frequently  are:  fOMng,  feedback, 
and  debriefing,  ^ese  are  naoes  we  have  given  to  training  techniques  which  play 
a  Central  role  in  the  program.  The  techniques  will  be  e3^lained>  described,  and 
then  discussed  briefly. 

TOR,  T,  0,  R,  -  is  an  abbreviation  for  ‘Training  derations  Report."  This  phase 
denotes  the  specific  information  gathered  by  observers  on  the  crew's  performance 
during  the  training  exerGise. 

Why  gather  such  information? 

Knowledge  of  results  or,  to  phrase  it  more  accurately,  knowledge  of  the  discrepancy 
between  what  was  actually  accomplished  and  what  should  have  been  accocipllshed,  l8> 
in  the  opinion  of  most  training  e3q>ert8,  a  necessary  condltlcni  for  effective  training. 

Most  training  programs  are  so  set  up  that  such  knowledge  is  fed  back  to  the  trainee 
as  soon  as  possible.  Production  line  operators  need  similar  l]^ormation  in  order 
to  be  able  to  evaluate  their  performance  rationally  to  know  how  to  improve  upon  it. 
Unfortunately,  in  complejc  man-machine  systemsi,  the  operator  often  finds  it  difficult, 
if  not  Impossible,  to  get  accurate  objective  knowledge  on  the  results  of  his  perform¬ 
ance.  In  addition,  with  the  exception  of  major  breakdowns  or  serious  errors  which 
stand  out  for  all  to  seSi  knowledge  of  the  total  system  performance  is  also  at  best 
quite  global  and  non-specific.  Hence,  information  which  can  serve  as  the  basis 
upon  which  the  crew  can  rationally  decide  what  steps  It  should  take  to  Improve 
Its  performance  is  generally  lacking  or  Inadequate.  The  TOR  attempts  to  supply 
the  necesscury  infonnatlon. 

How  do  we  TOR,  and  what  Is  TC^ed? 

This  varies  with  the  system.  As  you  already  know,  we  have  had  experience  In 
training  two  types  of  systems:  The  manual  air  defense  system  and  SAGE.  The 
foziner,  the  manual  system,  is  the  familiar  radar  site  ^eh  is  manned  by  small 
crews  of  twelve  to  fifteen  pers^ui  who  function  in  close  Interaction.  The  latter, 
the  SAGE  system,  la  radically  different.  It  is  hlfd^y  ecaputerleed  end  automa¬ 
tized  and  la  mipned  by  craws  of  iqp  to  100  persons.  In  addition,  since  the  SAGE 
crew  la  located  in  eeveral  large  rooms,  much  of  the  Interaction  between  the 
mambera  is  mediated  by  the  ccoputer  an^or  telephonas.  Therefore,  although  the 
basic  principles  undsrlylng  TORing  are  the  same  for  both  these  •yitems,  we  have 
f ovixid  it  neeeasazy  to  introduce  many  changes  In  specif Ics  when  we  begim  to  train 
the  SAGS  system. 

Here  too,  as  in  cdmost  every  other  aspect  of  the  training  program,  experience  ^ 

hamnared  hone  a  simple  basic  fact.  Ho  matter  how  correct  training  principles 
and  the  training  design  generated  by  them  are,  they  cannot  be  sppiled  slnply  to 
a  specific  situation.  A  gap  will  always  sadst  betwean  the  princlplea  and  prac¬ 
tice.  It  If  the  Job  of  the  professionals  In  the  field  who  ioplament  the  program 
to  fill  this  gap*  In  ether  words,  the  professional  In  the  field  does  not  passively 
Implement  the  training  program  but  creatlvaly  ipplles  it  to  fit  the  specific  condi¬ 
tions  of  the  crews  to  be  trained  in  light  of  the  problems  confronting  them.  He  is 
contlnuouily  called  upon  to  make  Judgments  based  upon  his  prof  esslpnal  skllli  in 
analymli  and  prognosis .  This  point  may  seem  ob^douf  and  you  msiy  wonder  w^  mention 
It  at  all,  let  alrae  stress  it.  Heverthelesi  It  is  quite  surprising  to  find  that 
all  too  frequently,  this  phase  of  "application"  is  overlooked  either  coopletely 
or  in  part. 
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Tp  return  to  how  we  lORed.  fORing  Is  rathei*  slfl$ld  In  the  manual  system.  The 
!]^R  tsam  vhlch  collects  information  on  the  crew's  performance  consists  of  two 
or  thi'ps  persons,  generally  an  e]Q>erlenced  noA-eommissloned  officer  with  one  or 
more  airmen  who  are  not  necessarily  experienced.  They  sit  in  full  view  of  the 
plotting  hoard  which  serves  as  the  central  memory  for  the  system.  By  ohservlng 
it  the  TOR  team  can  see  how  the  crew  processed  eeich  liiputted  flight  and  when 
each  atep  took  place.  In  addition ,  the  TOR  team  has  special  aids  showing  the 
flight  paths  of  the  planned  critical  flights  that  the  site  has  to  process  In 
a  known  manner  to  achieve  an  efficient  air  defense.  At  the  end  of  the  exercise 
the  TOR  team  has  a  record  of  how  the  crew  processed  the  critical  flights  and 
relates  this  processii^g  to  the  actual  simulated  li^ut. 

In  a  major  difference  In  TORlng  is  the  fact  that  information  gathering  on 
crew  performance  Is  automatized,  l.e.,  the  computer  maintains  a  record  of  the 
crew  actions  which  it  can  print  out  at  the  end  of  the  exercise.  This  information 
is  detailed  and  quite  long  ainee,  commensurate  with  the  Increased  size  of  the 
Crew  and  Increased  efficiency  of  the  processing  System,  the  number  of  critical 
flights  processed  hy  the  SAGE  sector  is  quite  large.  Ibe  information  available 
from  the  prlnt>outs  is  consequently  much  too  detailed  for  immediate  feedback  to 
the  crew.  Too  much  information  feedback  gives  Indigestion. 

The  TOR  team  for  SAGE  comprises  senior  experienced  personnel  who  are  recognized 
.hy  all  as  being  experts  In  the  system.  Their  Job  Is  two^fold.  During  the 
exercise  they  sit  among  the  crew  and  act  as  trouble  shooters,  noting  dovm  that 
which  they  consider  important  to  feed  hack  to  the  crew.  After  the  exercise  they 
can  also  examine  the  printouts  and  c\ill  from  th^  objective  Information  which 
they  cpasldsr  Isportant.  The  Information  that  Is  ga'^ered  and  selected  by  the 
TOR.  team  is  presented  to  the  crew. 

Because  of  lack  of  time  It  is  izzpossible  to  go  Into  details  on  the  difficulties 
which  we  ran  Into  In  setting  up  the  SAGE  program  despite  our  explicit  planning 
to  anticipate  the  new  environment.  Suffice  It  to  say  that:  (aj  The  original 
planning  left  something  to  be  desired,  and  (b)  because  both  training  per«' 
sonnel  and  the  crews  came  to  SAGE  from  the  manual  aystam,they  carried  over  with 
(them  numerous  habitual  ways  of  doing  things  that  were  Inappropriate  to  SAGE. 
We;hs4  quite  a  lot  of  trouble  at  the  outset.  This  is  being  ironed  out  with 
.4^erlsnct. 

Faedbaok,  as  we  use  it,  refers  to  the  technique  whereby  knowledge  of  results 
is  inpartsd  to  the  crew.  One  principle  underlies  feedback  it  has  to  be  an 
objective  aoeount  of  faettwl  performance  and>  by  Implleatlbn,  it  must  be  accu<^ 
rate.  According  to  this  view,  the  feedback  of  Inaccurate  information  is  worse 
than  no  foxmal  presentation  of  knowledgs  of  results.  The  nature  the  Inf  or « 
matioQ  fed  back  does  much  to  deterndne  whsther  t^e  subsequent  debriefing  will 
be  a  good  one,  that  is,  will  be  teak  and  problam  solving  oriented,  or  will  be 
a  aettlag  where  a  lot  of  steam  is  let  off  to  no  consequence. 

Why  this  stress  on  'objectivity"  and  "factuality"? 

Da  our  culture  one  person's  telling  another  that  he  is  performing  poorly  is 
^tsn  psresivfd  as  personal  criticism.  This  evokes  difenaive  behavior  \Aich 
hinders  the  indivUhmls  coneemed  from  facing  reality  and  gmaerates  inatead  a 
quest  for  iospegoats,  rationalisatioDs,  and  oft^  counter  attacks.  On  the 
other  head,  our  culture  has  a  very  healthy  rtspeet  for  facts  with  which  one 
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is  not  Buppoted  to  argu«.  If  erev  IB  convinced  that  they  are  Confronted 
with  facta  >  it  hecgneB  difficult  for  them  to  avoid  facing  l^e  IBBue  and  it 
beCoaiea  eaaier  for  the  dehrleflng  leader  to  eounter»act  dafenalve  behavior 
which  IB  apt  to  occur. 

Formal  organized  TORlng  and  feedback  are  but  atepB  that  lead  to  debriefing. 

In  fact ,  under  certain  conditionB  'Uiey  are  not  neceBBary.  aince  crewa  can  them-^ 
aelvei  preaent  the  neceBBary  infoziBatlon.  debriefing  la  the  Betting  and 
the  vehicle  vherein  and  whereby  the  moat  effective  crew  training  takeB  place. 

AB  Buch>  debriefing  could  be  a  aub Ject  for  a  aympeBium  ai  long  aa  thia  one  in 
ita  own  right.  Within  the  limltatlona  of  the  preaent  aynpoaiua  the  moat  I  can 
hope  for  ia  to  preaent  a  bare  akeleton  outline  of  what  ahould  be  richly  dia« 
cuaaed  in  itself. 

A  "good"  debriefing  waa  just  characterized  aa  being  taah  and  problem  solving 
oriented.  In  the  diacusaion,  the  individual  crew  rnttSbera  should  not  be  evalu¬ 
ated,  but  ayatem  perfozinanee  and  crew  procedures  should.  Pr^oaed  procedurea 
aimed  at  improving  ayatem  perfonaance  should  not  be  iapoaed  by  fiat  of  even  a 
large  majority.  The  debritf ing  leader  ahould  see  to  it  that  all  individuals 
affected  by  a  given  problem  ahould  have  their  say  in  diaciaaaing  it.  The  solu¬ 
tion  to  be  reached  should  be  one  that  all  involved  are  wii-Iiwg  to  try  out. 

Debriefing  la  rather  ainpie  in  the  manual  ayatem.  innediately  after  the  exercise, 
the  crew  meets  in  a  apecie^  room  or  area  set  aside  for  the  purpose,  ibe  TOR  ia 
presented  to  them  in  a  straightforward  manner.  Instances  of  poor  performance 
are  identified  cmd  a  discuasion  is  developed  to  determine  what  caused  the  poor 
performance  end  what  steps  can  be  taken  to  see  that  it  dpean't  happen  again. 

The  erew  mamhera  ara  also  encouragad  to  raise  additional  problems  concerning 
the  system  operations  which  they  want  discussed.  One  meaber  of  the.  crew  keeps 
a  "dsbrlefiag  .log"  In  which  all  the  problans  discussed  and  the  deoieions  taken 
caooemlng  them  are  recorded.  At  ^e  end  of  the  meeting  the  log  ie  read  back 
to  the.  crew. 

We  have  mat  with  many  difficulties  in  Implementing  debrieflnge  in  the  SAGE 
systeB.  and,  at  present,  we  are  still  ftruggllng  with  the  matter.  Some  of  the 
madn  oauses  for  this  are:  the  large  eite  of  the  SAGS  crew;  the  lack  of  adequate 
debrief ingi  •pace;  the  ceavlead-ty  d  SAGS  performance  which  makes  It  difficult  to 
give  jtmidwledge  of  resulte  that  is  rslevant  to  all  the  crew  mastberej  and  a  marked 
decreMe  in  the  iDterdependenee  of  the  crew  mailers  beoause  of  the  oamputer's 
mediation  of  much  of  their  interaction.  These  difficulties  seem  to  be  technical 
and  bam  not  led  us  to  dpUbt  the  basic  utility  of  debrlefl;^  aw  a  training  teeh'» 
nlque.,  However,  va  now  know  that  debriefing  ie  much  more  difficult  to  apply  to 
•cmm  men-aMohiae  syateme  than  to  othara,  eapaoially  when  these  syatemf  are 
deeifaed  without  training  coaiidefatioas  being  tikan  into  aooount. 

Tha  paV'rdffi  of  a  program  with  "good"  debrief iniS  ere  aaiigr.  Below  aope  a  liet  of 
six  of  the  more  pbvloua  ones: 

1.  quality  of  t|£  solutions  found  ^  problaais  confronting  the  crew  ^  euperior. 

In  these  matters,  more  brains  representing  a  greater  eraa  of  collective  e!^ri» 
enee,  is  better  thaw  leas  brains.  The  fact  that  all  those  who  will  be  affected 
by  the  proposed  solution  are  given  a  chance  to  have  their  say  concerning  it, 
at  least  guarantees  that  it  will  not  contain  the  usual  nusber  of  "bugs"  that 
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steb  from  unique  contingent  conditions  wMch  es^^rts  have  no  vay  of  knovlng. 
For  Instance >  the  absence  of  a  prescribed  Coianunlcatlon  channel  cannot  be 
known  by  the  eiQert  but  is  certainly  imowa  by  the  operator  \^o  mans  the 
position.  He  will,  therefore,  see  to  it  that  no  proposed  solution  is  based 
on  this  nou'^exlstlng  telephone  channel. 

2.  quality  ^  solution  inplenention  is  superior.  Ibis  for  two  reasons* 

Men  un^rstand  a  procedure  far  better  if 'they  ^scuss  and  ansdyze  it  than 
when  It  is  handed  to  them  in  a  written  memorandum  no  matter  how  precisely 
phrased,  what  men  understand  better  they  cem  l^lement  better,  in  addition 
men  are  more  highly  motivated  to  implement  a  procedure  which  was  formulated 
With  their  active  participation. 

3.  Early  rectification  of  many  nrobl^  before  they  beccaie  critical.  This 
occurs  both  overtly  and  covertly.  Because  of  %ne  atmosphere  in  a  "good" 
debriefing,  men  often  raise  problems  which  in  more  formal  circumstances 
they  Would  hesitate  to  mention.  And  this,  in  tuxn,  can  play  the  role  of 
the  stitch  in  time  which  saves  nine.  In  addition,  the  debriefing  often 
has,  willy-.nilly,  a  "therapeutic  effect".  Ihe  mere  fact  of  disGUSSlng 
operational  problems  with  other  crew  menbers  also  tends  to  use  a  clinical 
term,  to  "work  throxi^"  problems  in  interpersonal  relations  that  exist 
among  the  crew  members  before  overt  conflicts  develop.  And,  last  but  not 
least,  a  procedural  solution  reached  by  discussion  will  frequently  include 
In  a  sort  of  piggy-back  manner,  solutions  to  v^ious  other  minor  problems 
that  are  Irritating  the  crew. 

4.  A  Series  of  debriefings  ^  an  excellent  educative  setting  ^ere  the  grew 

ijsAerS  get  to  know  algt  aSput  hHe  in  general,  in  discussions 

which  perilin  to  proposai  ^d  Justlf icaiion  of  sj^iem  operating  procedures 

a  lot  of  Information  concerning  the  overall  nature  of  the  system  is  elicited 
Since  the  crew  membera  are  Involved  In  the  diseusslon,  they  tend  to  remember 
the  Information.  Increased  knowledge  of  a  system  by  the  operators  of  a 
system  is  always  a  worthwhile  thing. 

2ZS£||^  morale  of  ^e  crew  laproves.  A  group  of  men  who  iuccetef^Lly 
together  gad  e^ve  prohleeii  together  get  to  know  each  ether  relatively 
Intimately.  This  results  in  a  developiBnt  of  an  esprit  de  cozp,  ea  laereese 
la  group  coheslveaess.  It  follows  that  the  overall  morale  will  rise.  Coa- 
comitaat  with  this  developmeat,  there  Is  aa  iaereesiag  canpatlbllity  ot 
ladlvldual  crew  member  goels  aad  the  system  goals,  that  ls>  the  product 
that  the  eystem  was  deeigaed  to  process  beeones  laportant,  per  se>  to  the 
crew  both  as  a  group  and  as  Individuals. 

6,  And  finally,  there  emerges;  A  trained  effftcleat  ^blem  sel>^.ag  group  well 
prepsred  cope  wltt  ifflaiBPeclei  cMt^i^ieer  wbat  ve  have  hid  to  W 
hSreral^lag,  fee3FeiSk7vi”1ESbHeH^"’lSr¥eea  a  result  of  our  experlrace 
in  implemeatlag  the  System  Trelxilag  Program  aad  la  experlmeatatloa  both  la 
a  li^oratory  and  uader  field  coa^tlons. 
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Preceding  papers  have  Indicated  the  nature  of  the  System  Gaining  Program  (STP}> 
problems  relative  to  Simulation^,  and  Some  of  the  major  ooneepts  related  to  the 
use  of  knowledge  of  results  in  debriefings.  The  present  paper  will  dlseuss 
three  of  the  major  problems  arising  from  the  conduct  of  a  program  of  this  nature 
In  the  military.  The  System  training  program  was  deelgned  to  be  operated  by 
personnel  of  the  Air  Defense  Command  after  orientation  by  the  Staff  at  S9C. 
Originally  the  SDC  representative  at  each  military  headquarters  advised  the 
Congoander  and  performed  a  liaison  function.  At  present  the  field  training  team 
la  much  more  active  In  the  conduct  of  the  STP. 

Some  of  the  problems  we  found  are  unique  to  STP  while  ethers  will  be  found 
ii^erever  there  are  conditions  similar  to  those  with  which  we  work.  Major 
problem  areas  which  might  be  mentioned  Include  those  due  to  (1)  Introducing 
new  procedures  Into  a  culture^  (2)  utilizing  military  peraennel  as  training 
personnel^  and  (3)  aspects  of  the  military  culture  antagonlatlc  to  new  training 
procedures.  I  will  present  these  in  that  order. 

As  an  exanple  of  problems  arising  from  introducing  new  training  proceduree 
which  conflict  vlth  goals  and  activities  already  present,  let  ua  discuss  the 
debriefing  aspect  of  the  syatem  training  program.  The  debriefing,  as  we  know 
it,  vlth  an  entire  crew  participating  In  group  problem  aolvlng,  is  foreign  to 
military  culture  in  many  respecta.  Many,  particulariy  the  lower  echelon  officers 
who  bacame  the  debriefing  leaders,  perceived  the  debriefing  situation  as  an  abro« 
gation  of  responsiblllty-^a  situation  in  which  the  responsible  officer  relinqidshed 
hli  leadership  function  to  persona  of  lower  rank.  STP  does  liOT  ask  the  responsible 
officer  to  step  aside  only  to  utilize  resources  of  the  entire  crew.  However,  this 
erroneous  perception  created  special  problems  for  the  training  persoBnel. 

In  addition,  officers  willing  to  take  the  special  kln('  of  leadership  role  often 
could  not  do  so  because  of  personal  considerations  or  other  demands  upon  their 
time  which  led  to  conflict  of  interest.  In  order  to  Institute  the  most  effective 
debriefings,  the  part  of  STP  most  at  variance  with  the  militaory  culture,  It  was 
necessary  that  certain  aspects  of  the  military  culture  beeoM  cdtered. 

Frequently  Institutions  attempt  to  force  a  change  by  the  issuance  of  directives. 
Mhlle  reguletlons  and  directives  are  necessary  to  deserlbs  ebangss  and  to  estsb* 
liih  pexttisslon  to  adopt  new  procedures,  they  are  not  sufficient,  ^en  when  the 
directed  person  Is  willing,  eoopllance  to  a  regv|[Latlon  may  be  laposelble  If 
other  eoadltlons  are  not  conducive  to  the  changM.  As  Lewln  has  In^cated,  one 
cannot  change  one  aspect  of  a  cultiure  without  bringing  about  concurrent  ehanges 
In  related  aspects. 

We  find  that  adoption  of  debriefing  proceeds  best  when  two  general  condltloas 
exist: 

1.  First,  visible  evidence  of  coigasnd  support  paist  promote  good  STP.  Whenever 
the  ComuMadlng  Officer  desires  something,  military  personnel  att^t  to 
cooply.  This  Is  the  place  for  directives  In^cating  what  should  be  accomo 
pllshed.  But  beyond  this  we  need  other  evince  of  support.  Active  and 
continiUng  interest  in  the  program  seem  to  be  accompanied  by  interest  and 
activity  by  operating  personnel. 


2.  Second^  the  orgahlzatloh  of  station  activities  iniist  permit  coi^llaiice. 
nils^  In  a  sense ^  requires  a  modification  of  the  culture  of  the  organi¬ 
zation.  Often  dehrlefings  are  degraded  hy  other  activities  \^ich  compete 
for  time.  For  instance,  a  schedule  of  exercises  near  the  end  of  a  shift 
causes  conflict  hetween  cleanup  duties  or  chov  time  on  the  one  hand  and 
dehrlefings  on  the  other.  Other  tasks  interfere  with  STP;  for  example,  a 
coincident  live  mission  may  cause  cancellation  of  all  or  part  of  the  train¬ 
ing  mission- -most  often  the  dehriefing  suffers  as  it  is  under  the  control 
of  the  local  officer,  particularly  the  dehriefing  leader  vho  detenttinea 
its  length  and  its  content. 

The  point  I  wish  to  make  here  is  that  when  personnel  perceive  a  specific  aspect 
of  a  program  or  the  entire  special  program  such  as  STP  as  in  conflict  vlth  the 
culture  or  with  the  on-going  activities,  the  program  usually  is  degraded.  We 
find  some  of  the  best  dehrl^lng  practices  at  installations  where  the  traixilng 
person  finds  a  commander  who  is  receptive  to  this  aspect  of  the  program. 
Activities  are  modified  to  support  it.  In  these  places  debrief Inge  are  held 
in  spaces  especially  set  aside  for  them  with  plenty  of  time  allowed  free  of 
operational  duties. 

Another  aspect  c  promoting  desired  dehriefing  hahavlor  is,  of  eouree,  the 
training  of  the  leader.  This  training  must  deal  with  hie  attitudes >  hii  percep¬ 
tions  of  the  program,  and  his  skill.  He  muat  come  to  aee  that  the  debriefing  if 
a  tool  for  fulfilling  his  responsibilities  to  improve  performance  of  his  crew. 
Once  he  sees  the  task  in  this  way,  becomes  convincad  of  coomand  Support,  and  has 
time  to  do  a  good  Job,  the  traini^  task  becomes  much  more  sipaple. 

The  second  problem  area  relates  to  utilizing  military  personnel  as  trainers. 

By  way  of  introduction  I  will  discuss  the  adaqx^y  of  SOP's  as  means  of  ragu« 
latlng  performance.  Throughout  the  military,  Standard  Operating  Proceduree 
are  formulated  for  the  purpose  of  implementing  directives.  These  SOP's  attempt 
to  prescribe  the  ways  to  proeeea  information  or  to  execute  othar  taske.  We 
find  that  different  proceduree  often  become  effected  whenever  the  Standard 
Operating  Procedure  is  not  adequate  for  local  laplamentatipn,  whan  the  local 
personnel  are  unable  or  unwilling  to  execute  than,  or  when  the  SOP  la  not  under¬ 
stood.  When  the  SQP  le  worded  explicitly  and  stated  in  terms  of  actual  oper¬ 
ations,  less  formulatlpn  of  special  Ipced  procedures  peciu's.  For  axample, 
individuals  adjust  to  overload  in  a  variety  of  ways  such  as  ooadslnlng  categories, 
filtering,  and  reassigning  priorities  to  tasks.  Unless  the  permleeible  adjust- 
ments  are  made  known  and  the  contingent  conditions  for  their  inplenentatlon  are 
spaclfiad,  each  person  will  filter  or  ecnibine  on  the  bails  of  his  own  percaptlpn 
of  the  teak.  People  also  taka  ehort-outs  In  task  parfonunct;  unlaif  thaaa  are 
known  to  pthara,  systam  performanea  miy  break  down  bacauaa  each  one  is,  in 
effect,  working  by  his  own  rulaa.  As  indicated  in  an  earlier  paper,  debrief inga 
prpvida  an  opportiuilty  for  these  local  precadures  to  be  fomilated  and  for  an 
exchange  of  Information  about  peraonal  operating  prooedxurcs.  In  iplte  of  this 
Opportunity,  ve  find  conflicts  resulting  from  uxiaxpeoted  behavior,  omlsBlona, 
or  non-congruence  between  behavlora  of  different  perapns. 

The  preceding  paragraph  has  lapllcatlons  for  the  conduct  of  a  training  program, 

As  Rogers  mentipned,  in  STP  some  of  the  simulation  is  effeotad  through  auto¬ 
mation  but  men  do  much  of  it.  Furthermore,  in  STP  thi  training  personnei  pro¬ 
cess  input  and  performance  data.  Thus ,  in  ad^tien  to  the  eyatm  being  trained 
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(cay.ed  th0  operating  extern) ,  we  have  the  ejrstem  doing  the  training  (called 
the  training  eyetem) .  ^e  eeme  problems  of  obtaining  confofTaance  to  regulations 
foxind  in  the  operating  system  are  present  in  the  training  system. 

The  SOP'S  Of  the  training  system  must  be  at  least  as  detedled  and  as  carefully 
formulated  as  those  for  the  operating  system'.  True,  some  areas  of  training 
require  flexibility  and  similator  response  to  the  developing  situation.  However, 
other  tasks  cannot  permit  this.  When  machines  make  the  routine  inputs,  the  prob« 
lem  of  exactness  lies  in  computer  progrannlng.  But  when  people  must  handle  the 
inputs  or  data  collection,  unique  personal  operating  procedures  cannot  be  allowed 
except  in  very  minor  matters  which  don't  make  any  difference  in  the  timing  or  the 

nature  of  the  input.  ^ 

» 

In  short,  a  training  program  is  being  planned,  every  bit  as  much  attention 
should  be  given  to  the  procedures  of  the  personnel  in  the  training  system  as  to 
the  procedures  of  the  operating  System,  or  to  the  mechiaies  of  simulation. 

Essential  aspects  of  this  part  of  -j^e  program  include  specification  of  desired 
inputs,  determination  of  allowable  deviations,  formulation  of  training  SOP's, 
training  of  the  training  crew,  and  monitoring  of  its  work. 

The  third  problem  arises  from  the  fact  that  certain  aspects  of  the  military 
culture  are  antagonletlc  to  the  training  program,  be  it  STP  or  some  other. 

One  of  the  prime  sources  of  difficulty  lies  in  the  turnover  of  pereonnel  in 
the  services,  ia.  some  radar  crews  $0  per  cent  of  the  aesiberB  will  have  been 
replaced  within  a  year.  We  probably  should  consider  a  crew,  not  as  a  tsam  of 
specific  men  working  together  in  a  co-ordinated  fashion,  but  as  an  organization 
with  a  slowly  changing  culture  of  its  own. 

When  turnover  occurs  the  new  man  does  not  know  those  unique  procedures  of  the 
crew  to  which  he  is  assigned.  The  replacement  may  be  Just  out  of  basic  training 
and  thus  relatively  naive,  or  he  zoay  have  transferred  from  another  station  bring* 
ing  with  him  a  culture  somewhat  alien  to  that  which  he  now  contapti.  in  either 
caae,  each  transfer  requires  a  period  of  orientation  and  initiation. 

When  a  number  of  crewmen  work  together  for  a  time  a  great  deal  of  "social 

Isezhing"  occurs.  As  a  result,  a  stable  crew  may  have  less  nssd  for  formal 

SOP'S  than  a  newly  formed  crew  or  one  which  suffers  fzmqusnt  Insult  to  its 

inteipilty.  The  crewmen  laam  each  other's  idlpsynoranies  end  can  eceoBapdate 

or  compensate.  In  a  number  of  field  and  leboratpry  situations  we  have  noticed 

a  profpresslon  in  crew  development  from  eonoem  with  one's  own  prpoedures  through 

an  interest  in  luadsrstanding  other  men's  setlpns,  to  trying  to  modify  the  others' 

proosdures  and  ^en  to  the  point  where  the  men  h^  each  other  by  sharing  the 

load.  Preaumebly  with  conatent  turnovar,  tha  highar  levala  of  parfonMnoa  would  i 

not  ba  attalnad. 

We  had  noticed  that  oparatiag  orewa  in  the  field  saldon  axcaadad  a  certain  laval 
of  parfoxnanoa,  regardless  of  the  length  of  the  training  period.  On  the  other 
handf  when  man  were  assigned  to  qur  •imulation  laboratory  vhieh.  duplicated  a 
radar  station,  they  learned  to  work  a#  a  crew  and  functionad  at  fairly  high 
levala  of  parfprmance  efW  only  a  few  waaka  at  training.  Wa  baliaved  that  one 
causa  of  the  dlaparlty  lies  in  the  differenees  in  turnover.  Consequently,  a 
serlee  of  laboratory  studies,  utilizing  an  analogoua  taak,  vaa  izuitituted  to 
axamine  this  hypothesis,  and  to  explore  the  factors  eontributing  to  the  phenome¬ 
non  while  trying  out  trednlng  procedures  which  mi|ht  oounteraot  the  effects  of 
turnover  shovild  they  be  identified. 
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la  the  flrit  a^efiment,  memhera  of  a  erev  trained  together  ao  that  all  mafljera 
poaaeaaed  e^ual  eicperlence.  Peraons  ai^ericneed  on  another  erev  then  apatam^ 
atieally  replaced  original  erev  infers.  The  perfonoance  in^rovenant  cis^e 
flattened  out  vhen  contpared  to  the  perfonaanee  of  a  erev  not  turned  over* 

in  a  aeeond  experiinent,  ereva  coiqpoaed  of  neabera  with  a  range  of  ei^erienee 
vere  replaced  by  green  or  untrained  people.  In  thia  caae  we  found  a  deeramant 
in  perfor&anee. 

Further  analyaia  of  the  data  with  reapect  to  the  ayetern  enployed  pinpointed 
aeoe  of  the  eauaea .  "Greenneas "  or  inei^erienee  ae  was  eonf  ounded  with 
poaition  on  the  erev.  in  tieie  ayatem  aome  paira  of  nan  worked  face  to  face 
and  eonnunieatlon  between  them  ve^  limited;  theae  men  did  not  pace  one  another. 
Men  in  anol^er  group  of  poaitiona  communicated  over  the  telephone  and  in  each 
pair  one  peraon  paced  the  other  by  paaaing  inrormatlon.  The  bulk  of  the  de^ 
grading  effect  of  turnover  vaa  obaerved  in  thoae  caaea  in  vhieb  eotmunieotion 
waa  by  telephone,  with  one  peraon  pacing  the  other. 

Other  effecta  were  obaerved  with  increaae  in  the  proportion  of  men  replaced. 

In  the  debrief Inga,  evidence  of  group  eoheaiveneaa  vaa  aeen.  Apparently 
introduction  of  only  one  man  to  this  crew  of  aeven  did  not  affect  the  eoheaion. 
But  vhen  three  or  more  rnembera  of  the  crew  were  replaced,  cohesiveneaa  ae^d 
to  deteriorate  and  auh-groupinga  occuirred^^-there  aeemed  to  be  a  breakdown  into 
new  veraua  old  men  with  respect  to  advocacy  of  procedurea. 

Several  suggeatlona  for  dealing  with  this  matter  of  turnover  come  to  mind.  The 
moet  obvious  would  be  reduction  of  turnover.  One  method  might  be  the  organic 
zation  of  crews  of  men  who  entered  training  at  the  same  time,  trained  together, 
and  transferred  or  discharged  em  a  groi;p.  This  haa  bean  attmopted  in  aome  of 
the  services,  I  ui^eratend. 

A  commander  could  minimize  transfer  among  creva  on  hla  atatlon^^lt  might  be 
better  for  a  crew  to  be  shorthanded  than  to  get  a  temporary  replacement  from 
another  crew.  When  replacements  are  neceassry  they  might  be  assigned  at  flrat 
to  Jobs  having  lees  effect  upon  the  work  of  others. 

Designing  training  programe  epecifically  to  counteract  the  effecti  of  turnover 
ilao  eeema  to  be  a  rather  obviovis  solution.  Frequently  ve  have  seen  the  effect 
tiveneaa  of  STP  in  bringing  "green"  men  up  to  levela  of  perforDonee  which  perndt 
them  to  function  well  as  crew  maibers.  chie  training  procedure  tried  in  the 
turnover  mqperiments  mentioned  above  conaiated  of  aupplying  poiltional  problem*, 
orientad  feedback,  That  is,  in  addition  to  giving  data  regarUng  total  perform*’ 
ance  to  the  crew,  the  new  men  was  supplied  with  information  relative  to  the 
^fleudties  he  appeared  to  be  having,  ihia  change  in  feedback  procedures  led 
to  narked  improvement. 

I  have  dlecussed  three  problems  of  conducting  ayatem  training  In  a  military 
cud^e.  I  submit  that  they  will  be  with  us  for  tome  tiaie.  On  the  other  hand, 

I  believe  that  these  difficultlea  need  not  necessarily  vaaken  a  training  program. 


'THE  lISlAlMiNS  FHOBIiEMS  OP  FtJTiniE  SYSTEMS" 
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Let  ine  first  review  the  model  of  the  system  tralhlhg  program  to  see  what  are  Its 
probable  streogths  and  veakhesses.  Ihis  model  vas  develc^ed  for  a  aanual  data 
processing  system,  iiext^  let  me  review  some  of  the  lessons  we  have  learned  in 
its  Initial  applleatlon  to  a  8emi>au.tomated  data  procesaing  system,  namely, 

SAGE.  Then,  I  Will  disctiss  Some  of  the  problems  which  confront  humans  in  the 
SAGE  environment.  Finally,  l  will  siAggest  a  strategy  for  developing  training 
for  the  problems  of  future  systems. 

YOU  will  recaOJL  that  the  STP  model  consists  of  basically  four  parts.  First, 
there  is  the  simulation  of  Inputs  and  portions  Of  the  environment.  Second, 
there  is  the  system  operation  in  the  context  df  the  simulated  inputs.  Third, 
performance  resvilts  relating  to  the  effectiveneis  of  the  ayetem  and  certain 
othar  specific  kinda  of  performance  are  reported  to  peraonnel  in  the  ayatam. 
Finally,  there  is  the  debriefing  where  several  or  all  peraoni  in  the  ayetem 
meet  to  diacuae  the  results,  identify  problems,  and  devise  solutions. 

What  are  the  strengths  and  weakneasea  of  the  model?  First ,  let  ua  examina 
three  of  its  major  asaumgptions.  The  first  aSaumptlon  is  that  many  major  train¬ 
ing  problems  exist  In  and  therefore  should  be  remedied  in  a  ayatmn  context.  This 
has  at  least  two  advantages.  First,  to  the  extent  that  the  behavioral  inter¬ 
actions  are  a  part  of  the  training  problem  it  is  ixoportant  to  devise  solutiona 
which  take  these  interactions  into  account.  !&ls  might  be  raf erred  to  Si  train¬ 
ing  in  a  ayatem  context.  Second,  it  can  be  shown  that  changes  in  oparatipnal 
procedures  or  equipment  in  one  part  of  a  ayatem  often  requlra  ohangea  in  other 
parts  of  ^  ayatam.  This  ia  conceptually  similar  to  the  coneapt  of  ehanga 
diacuaaed  aarliar  by  Dr.  Jensen.  The  obvloui  dlaadvantagti  are  that  to  era^ 
up  a  ayatam  or  a  portion  of  a  iyatam  la  both  very  axpanslva  and  runs  the  risk 
of  loaa  of  control  of  the  trailing  situation. 

A  second  aaaimption  in  this  model  is  that  the  Information  raported  in  the  faed- 
baok  anphaaisas  task  ralavant  behaylora.  Hanca,  problama  of  motivation  and 
Individual  needs,  which  are  Important  In  man-machina  syatams,  ^aa  Indirectly 
In  this  kind  of  a  training  context. 

A  third  aaiunptlon  ie  that  -tihla  la  a  problem  solving  model  and  mora  apaelflcally 
a  group  problem  splvlDg  model.  Lqpllclt  in  this  la  an  eaphaali  on  the  responal- 
blllty  of  Individuals  in  a  group  to  raeognise  problama  and  to  contrlbuta  to  the 
aolution  of  these  problama.  This  is  an  advantage  to  the  extant  that  It  Inauras 
the  problama  which  are  identified  are  the  actual  problama  which  the  membara  of 
the  eyatam  have  ancountarad.  Also,  where  there  la  a  nseaaalty  for  individual 
cnanrltmant  to  action,  a  group  problem  solving  modal  aaama  to  hara  semta  auparl- 
orlty  In  this  raapact.  An  pbvioua  dlaadvantaga  la  that  It  may  run  epntrary  to 
the  noxna  of  the  oultura  In  wldeh  the  ayatam  la  aat.  G^paoinoai^,  the  npnpp 
cf  «  eultvffa  may  be  such  that  an  IndlviduAL  doaa  not  adimt  or  raoop^sa  proUam# 
or  that  he  does  not  taka  the  raq^onalbility  for  thaaa  problami  but  rathar  wmita 
for  word  from  a  higher  level. 

One  way  to  test  the  ganarallty  of  a  tralnisg  modal  la  to  avaluata  its  affaetlva- 
nasi  in  the  aattlng  of  a  aonawhat  lUffarant  type  of  datn  prpoafaing  ayiitaB.  Va 
are  beginning  to  aceummulata  acme  axparienea  in  this  rtapaet  as  va  avaluata  the 
application  of  this  model  or  variations  of  it  in  the  SAGS  ayatam.  You  may  recall 
that  the  SAGS  ayitam  la  a  aaml-automatle  ground  acnrlroamant  iyatam  which  has  aa 
its  eantar  a  hlgh-ipead  digital  coqputar.  Qperatort  ean  axtraet  data,  maka 
daolsiens,  and  instruct  the  coeputer  via  manuaO.  Intarvantioni  by  maani  of 
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assoeiated  consoles  during  the  real  time  operation  of  data  processing  steps.  It 
is  reasonable  to  assume  that  the  SAGE  system  is  similar  in  many  respects  to  future 
high  speed  data  processing  systems.  While  it  is  still  too  early  for  any  veiy 
thoroiigh  going  evaluation  of  either  this  training  model  as  applied  to  SAGE  or  of 
the  full  range  of  training  problems  encountered  by  cjrsm  in  a  SAGE  operation,  1 
believe  ve  have  enough  es^eri^ce  to  malce  a  fevr  preliminary  ■tatements  along 
this  line. 

First,  it  is  now  recognized  as  highly  desirable  to  allow  a  ahake«down  period  for 
the  equipment,  coag)uter  progr^,  and  tha  various  ^erationaj.  procedures  which 
are  required  to  run  a  system  ee  coopl«  as  SAGE.  I  think  this  is  analogous  to 
the  debugging  or  deghostlng  phase  which  applies  to  any  eo^llcated  piece  of 
equipment  in  Its  early  research  and  developmant  atagea.  initiation  of 
training  programa  with  the  expectation  of  eolid  training  outeomei  during  this 
shake^dovm  period  is  probably 'uznrealistic.  This  is  not  to  say  that  certain 
training  or  eimulatlon  proceduree  cannot  be  used  aa  part  of  the  shakedotm.  In 
fact,  the  aystem  training  simulation  program  has  been  used  as  a  inajcnr  shikedowi 
vehicle.  Any  system  which  undergoes  changes  can  benefit  from  such  a  simulated 
shakedown  vehicle. 

Second,  ^en  the  system  treiniog  model  Is  carried  forward  Into  a  eeml^ automatic 
data  processing  aystem,  the  scope  and  complexity  of  the  training  problema  becoow 
very  large.  The  problems  of  training  the  training  system  becoote  inteneifled. 

The  n\mfber  of  persons  required  ae  simulators  and  observers  increases  greatly. 

The  eoBgniter  programming  effort  for  the  development  of  data  retrieval  and 
analysis  programs  for  trednlng  purpoaea  becomes  a  major  activity.  With  this 
comes  the  long  lead  timee  req\^red  for  programming  smd  equipment  modification 
to  support  the  training  program. 

Ihls  leads  directly  into  a  third  observation^ '^namely,  that  there  develops  a 
rigidity  and  cooplexlty  in  the  automatic  Storige  and  data  processing  component 
of  the  training  system.  Training  problems  change  as  system  proficiency  chsoiges. 

There  is  a  problmn  of  providing  enough  adaptabiiity  or  flexibility  in  the  train* 
ing  program  to  provide  the  kinds  of  trsdnlng  wbleh  are  needed  at  a  particular 
time  or  point  in  the  development  of  a  system.  We  are  only  now  beginning  to 
learn  how  to  put  thla  kind  of  ads^gbility  into  a  eos^lex  trmLnlng  pro^am,  ans 
it  is  not  at  all  clear  yet  that  enough  capability  for  change  or  (adaptability  can 
be  built  into  a  se^^autcmated  trainlxig  program  to  Justify  the  resources  and  time 
required  to  develop  this. 

itod  a  final  observation  is  that  it  looks  as  thovigh  there  la  a  hierarchy  of 

Importance  of  training  problems  in  a  data  processing  system  and  that  ^is 

hierarchy  changes  as  proficiency  of  ^e  crew  increiuies.  From  actual  observa*  ^ 

tlon,  both  in  the  manual  and  SAGE  systsis,  a  change  in  hierarehy  of  training 

problops  like  this  seems  to  occur,  nils  is  a  ohange  from  problsim  ctf  working 

with  equipment  in  one's  own  position  to  problems  of  procedures  cmd  co«‘OrMnatlon 

among  various  parts  of  a  systmn. 

Sow,  so  far  I  have  tried  to  contrast  or  ccepsre  some  of  the  characteristics  of 
a  system  training  model  with  the  problems  of  making  it  work  and  i4th  the  kinds  oT 
T  training  problems  It  would  meet  In  a  8ami>autcmated  data  processing  system.  We 

f  are  beginning  to  accvnulate  some  observational  data  which  is  derived  from  a 

slightly  different  point  of  view  with  respect  to  problems  of  humans  in  systems. 

This  is  based  tq^on  actual  experience  in  operating  in  most  of  the  posltimui  In  a 
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SA:fE  Direction  Center  and  In  intensive  operation  of  a  paxiiicular  position  in  the 
weapons  section  in  a  system  context.  Based  upon  this,  we  have  identified  six 
problem  areas  faced  by  himans  in  this  kind  of  a  data  processing  system.  Ibese 
areais  are:  (1)  Individual  positional  skills;  (2)  Bdiavlpral  interactions  with 
Other  humans  in  the  system;  (3)  Awareness  of  effects  of  aetions  on  efficieney 
of  systw;  (h)  Motivation  for  task  or  goals  of  the  system;  (5)  Complex  decision 
making  in  an  automated  data  processing  environment;  and  (6)  Isolation  and  social 
motivation. 

Individual  positional  skills  refer  generally  to  the  skills  required  to  ccmBunicate 
with  the  computer.  *^680  skills  Include  those  necessazy  to  extract  and  interpret 
information  and  those  required  to  insert  into  the  computer  instructions  or  actions 
which  the  operator  decides  are  appropriate  in  his  data  processing  teisk.  Typically, 
a  Computer  data  processing  system  reads  out,  prints  out,  or  displays  inforiDatlon 
to  operators  in  coded  or  symbol  form,  hence  the  operator  mult  learn  to  interpret 
the  codes  or  symbols  very  rapidly.  There  is  a  further  problem  in  some  systans 
of  many  clasees  of  information  with  the  possibility  of  ambiguity,  crowding > 
overprinting,  etc.  This  Is  peurtlcxilerly  true  If  information  is  ^Splayed  on  a 
situation  or  geographic  type  display.  Usually,  instructions  are  Inserted  manviedly 
into  the  Computer  by  means  of  sequences  of  steps  or  switch  aetions.  ibess  must 
be  learned,  and  there  is  considerable  motor  skill  in  performing  them  correctly 
both  in  aequency  and  tlmiz^g  with  respect  to  computer  cycling  or  frame  time. 

These  skills  might  be  thou^t  of  caccsptually  as  the  basic  skills  of  a  particular 
position,  because  without  them  the  operator  is  unable  to  comnunlcate  with  the 
Gcoputer,  vhieh  is  the  central  locus  of  data  procsssing  in  such  a  iystsm. 

Behavioral  interactions  with  other  humans  in  the  system  can  randfy  into  a  great 
many  types  of  prohiems.  However,  I  will  confine  myself  to  the  vexbal  coaounleations 
'Which  are  elicited  or  reqiiired  by  operators  as  they  psii'form  thslr  data  procsssing 
task.  One  aspect  of  this  problem  is  ^S  sheer  number  of  cooBunications  'Whleh  can 
converge  upon  certain  positions  in  a  data  processing  system.  In  fact  certain  key 
poaitlons  in  a  system  can  become  So  overloaded  OGnmnilcatlon  wise  that  it  is 
ijqppsslble  for  one  individual  to  respond  phyilcaliy  to  all  of  tbs  eoBBsi^catlon 
attespts.  In  certain  of  these  positions  the  operator  is  provided  with  an  assist-* 
ant,  referred  to  as  a  technician,  whose  responsibilities  are  to  assiat  in  aqms 
of  'the  data  extraction  and  manual  action  insertiozis  to  the  computer  and  to  assist 
in  handling  the  conmunlcation  load  both  to  and  from  the  position.  It  turns  out 
that  this  is  a  mixed  blsssing,  becav^e  a  new  set  of  cooBunicatlon  problaos  develop, 
nemely,  'the  meshing  of  communl cations  between  the  two  operators.  It  has  been 
observed  that  with  inappropriate  signals  and  inappropriate  timing  each  can  lnter» 
fera  with  the  work  of  the  other  while  attenpting  to  make  legitimate  ccnnunicatlens 
to  that  person.  I  have  seen  a  substantial  amount  of  positional  dsgrsdatlra  Intro^ 
ducsd  because  these  two  operators  would  cooBunleats  with  saeh  other  at  times  that 
interfered  with  some  other  task,  or  information  would  be  Pfwsed  to  the  other  vi'th 
the  assumption  that  he  had  received  It  when  in  fact  he  was  preoccupied  with  another 
task  and  did  not  receive  the  information. 

ANareness  of  effects  of  actions  on  the  efficiency  of  the  system  might  be  regiurded 
as  the  key  objective  of  system  trslning.  To  illustrate,  I  will  give  here  only 
one  exaaple  which  could  be  multiplied  many  times.  In  SAGE  the  Weapons  Asslgmsnt 
Officer  must  have  at  his  disposal  a  number  of  kinds  of  information  to  make  effee- 
tive  decisions  concerning  commitment  of  weapons  to  targets.  This  ii^ormation 
must  be  timely,  accurate,  and  complete.  Some  of  this  information  is  inserted 
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Vl.a,Ba9iSuaI  card  punchliig  In  a  room  which  Is  physically  separated  and  organl«a‘- 
..^ionally  remote  from  the  Weapons  Section  qf  the  center ^  ^e  operators  who  must 
.  Insert  this  Ij^ozmtlon  have  many  other  classes  of  Information  to  insert  and 
which  axe  extremely  in^ortant  for  various  other  tasks  or  functions  In  the 
syitem.  I  think  it  la  too  much  to  depend  upon  edmijalatratlve  directivea  or 
a  policing  ayatem  to  see  that  the  crucial  Information  la  provided  in  an  accurate 
and  timely  manner,  ^en  an  operator  has  saveral  johs  to  do  at  once  he  must 
Choose  among  them>  that  la,  he  will  have  a  aubjectlve  priority  scheme  from  which 
he  will  select  those  that  he  ought  to  do  first,  second,  etc.  Often  this  meana 
Some  things  don't  get  done  hecause  of  over  crowding,  ^at  is  important  here  is 
that  the  operator's  priority  hierarchy  conform  as  nearly  as  posslhle  to  some 
hierarchy  of  importance  which  will  render  maximum  system  effectiveness.  One 
of  the  ways  to  do  this  is  give  him  data  pertaining  to  the  conaequences  of  hla 
aetlona  or  lack  of  actions  upon  certain  crucial  performance  outputs.  Because 
.  these  decisions  have  to  he  made  In  relation  to  a  large  number  of  Items  and 
through  changing  clrexanstancea  >  It  does  not  seem  praetleal  to  dapend  solely 
upon  rules  or  supervisory  sxirveillanee  to  aceompllah  this  kind  of  performance. 

I  helleve  motivation  regarding  the  task  or  goals  of  the  system  will  become  an 
inereaalngly  important  problem  In  more  automated  data  proceasing  aystema.  We 
cannot  rely  solely  \;pon  a  written  or  formal  statement  of  the  task  or  goal  of 
the  system  to  serve  as  a  motivating  device.  First,  becaxise  of  increasing 
specialization  and  coiqpartmentall nation  in  these  systems,  it  becomes  difficult 
for  an  individual  to  place  his  particular  task  or  Job  in  a  context  which  makes 
sense  to  him.  Second,  many  of  the  data  processing  Jobs  ax's  not  the  most 
.  exciting,  and  when  pertTormed  day  in  and  day  out  and  month  in  and  month  out 
become  in  fact,  routine  and  boring.  I  gueaa  we  don't  have  to  belabor  the  point 
that  a  highly  skilled  operator  who  regards  his  task  as  senseless  can  contribute 
as  much  ot  greater  degradation  to  system  efficiency  than  will  an  unskilled  or 
new  man  or  the  malfunctioning  of  eq^llpment  within  the  system. 

Costplex  dselilon  making  in  a  data  processing  environment  is,  I  think,  a  very 
fertile  territory  for  research.  Since  these  complex  decisions  are  often  the 
most  crucial  outputs  of  data  prooessing  systems,  we  need  to  determine  w^t 
eharacterl  sties  of  a  data  processlz^  environment  permit  the  most  effective 
dacision  making  on  the  part  of  humans. 

I  have  saved  isolation  and  social  motivation  until  last,  because  Z  think  the 
last  shall  became  first.  I  believe  that  in  future  systems  these  problems  will 
assume  major  proportions.  It  has  already  been  mentipned  that  automated  systems 
tend  to  coapartmentellze  operators  into  very  small  units.  Combined  with  this 
is  the  degree  of  ettentlpn  and  lack  of  mobility  which  la  required  to  operate 
eff actively  in  tenoa  of  inputa  and  outputs  with  a  oonputer.  In  addition  thare 
may  ha  a  aense  of  lack  of  control  of  the  environmant  or,  in  other  worda,  that 
tha  ooqputer  la  running  thinga.  All  of  theae  factors  will  maka  aoeial  moti<i 
vatlon  a  mors  iiportant  problem  in  future  aystema.  I  believe  we  do  not  realize 
the  extent  to  which  we  depend  upon  the  rcactiooe  of  others  to  deriva  a  ■tnae 
of  apciel  validation  for  what  we  are  doing  or  the  appropriatenees  of  this  task 
or  ^t  goal.  Also,  there  ee^  to  be  certain  minlmuffl  conditioni  necessary  for 
social  intaraotion.  When  theee  coaditlonB  are  not  mat,  humans  will  angaga  in 
halMVlors  or  adjuatasnts  to  reatora  aoma  aanlmuB  sat  ^  conditiMup  for  thla 
intaraotion.  MMoh  of  the  so  called  non<^task  priented  behavior  in  formal 
organlzatlens  can  be  evlbsumed  under  thia  typa  of  phenamanon. 
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lillf  bMfj^  otttli&id  ioM  of  th*  klBdi  of  vlilch  I  oaft  §••,  vhat  can 

AoM  atMvt  tbmitt  Bern  of  tteaa  pMfbiaaa  oaa  Im  raMdlad  by  »f»tm  datiga. 
Sd^a  liwtliiiti  tmui  aagiQMiilac  nd  afitan  Aaalin  iitileh  takaa  Into  aocoust  tba 
fttU  %akan^oK«l  intafaetiona  la  apitiB  opexatlea.  Xatafaatiai^  anooghji  iAm 
Boat  q:ualif  lad  ampaanrlaora  talk  nBofot  akiliad  epazBtovt  la  tba  SdCtt  e^taB,  tha 
aoBiOa  thraad  of  akllla  appaara  to  b9  thoaa  mileli  la^va  gatttng  Bpoaid  ^ 
XiBltatlona  of  tha  avalpn^  or  tha  eoBputar  prograB'.  To  tJBi  aacUat  that  aoBa 
of  thaaa  llaltatloaa  oaa  ha  ohaesad  prollaBa  ean  ha  fwiadlad  hp  ayitBia 
radaalgn. 

titea  It  QoBaa  to  training,  I  haliava  tha  foaling  It  grefWing  that  zu>  ona  training 
prooadiira  alona  la  adaquato  tor  tha  hl^jr  ooBplaB  a^  ohMginig  training  naada 
of  Aataii  proeaaalng  ayatama.  Training  ahould  ha  daalipMd  to  aeeoiiliah  apaeifio 
tritniag  pnipoaaa.  Por  aoBa  puxpoaaa  training  ahoold  ha  BBall  aeaia,  that  la, 
not  tl^ng  np  aajer  aqulpBant  or  larga  portloea  of  a  ipitaB.  Zf  part  of  tha 
prehlaB  involvai  haharloral  intaraotlona  aBong  parta  of  a  apitaB  or  aBoag 
tBdivldttiLa  in  tha  ayatOBi  than  thia  haharloral  tntaraetloB  ihotdA  ha  Inoludad 
aa  port  of  tha  traladng  anrironnant  or  training  apitaB.  Thla  auggaata  that  in 
aoBa  eaaaa  tha  apatan  training  nodal  ean  ha  agpUad  to  a  aaallar  portlOB  of  tha 
apatoB,  paxhapa  tha  intaraotlona  anong  thraa  or  four  eparatora  or  at  laaat  a 
Mall  portion  of  a  apatan. 

on  tha  othar  hand  thara  ara  training  prohlana  vhloh  raqulra  tha  partlolpatloB 
of  a  larga  portion  of  tha  apataB  and  tha  davalopBant  of  an  aBtanaiva  training 
apatM^  alnulatlon,  autenatad  data  oolloetlon,  ate.  nara  appaara  to  ha  an 
Inrraraa  ralatlonahlp  hatvaan  aoopa  of  apatOB  to  ha  tralnad  and  fraquaney  of 
training  parioda.  Adaptahilitp  to  ohanglng  trainiimi  prohlana  la  faeilitatad 
vlth  tha  aadiuB  or  aBall^aeala  training  progran.  M  apatan  intagration  haooBaa 
a  Bora  tniportaBt  prOblM  in  iAo  daralopnnt  of  a  partieular  apatan,  larga  aenla 
trnlBiBg  prngr— I  vill  haaona  aora  inpertaBt. 
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Society  at  the  University  of 
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